Using Memory-Protection to Simplify Zero-copy Operations

High performance networks (e.g. Infiniband) rely on zero-copy operations for performance. Zero-copy operations, as the name implies, avoid copying buffers for sending and receiving data. Instead, hardware devices directly read and write to application specified areas of memory. Since modern high-performance networks can send and receive at nearly the same speed as the memory bus inside machines, zero-copy operations are necessary to achieve peak performance for many applications. Unfortunately, programming with zero-copy APIs is a giant pain. Users must carefully avoid using buffers that may be accessed by a device. Typically this either results in spaghetti code (where every access to a buffer is checked before usage), or blocking operations (which pretty much defeat the whole point of zero-copy). We show that by abusing memory protection hardware, we can offer the best of both worlds: a simple zero-copy mechanism which allows for non-blocking send and receives while protecting against incorrect accesses. To make things concrete, consider an MPI computation that as part of it’s operation maintains a distributed table. Each worker might have code like this: // map from int to a big blob hash_map table; PutRequest put; GetRequest get; while (1) { if (world.Iprobe(ANY_SOURCE, PUT_REQUEST)) { world.Recv(peer, PUT_REQUEST, &put;, sizeof(put)); memcpy(table[put.key], put.value); } if (world.Iprobe(ANY_SOURCE, GET_REQUEST)) { world.Recv(ANY_SOURCE, GET_REQUEST, &get;, sizeof(get)); world.Send(peer, GET_RESPONSE, table[get.key]); } } (Naturally each worker will probably be doing something else in addition to maintaining their table.) This is fairly straightforward, but inefficient code; the blocking send operation prevents the worker from doing anything else until the current get request completes. We’d like to switch to use the non-blocking (ISend) primitive for effiency. A first attempt at this might look like: list pending; while (1) { ... if (world.Iprobe(ANY_SOURCE, GET_REQUEST)) { world.Recv(ANY_SOURCE, GET_REQUEST, &get;, sizeof(get)); pending.push(world.ISend(peer, GET_RESPONSE, table[get.key])); } // remove any finished sends check_for_completed(&pending;); Of course, after running this (or if we’re smart, before running it), we realize that there is a possible conflict between our put and get requests. We can now handle multiple simultaneous sends, but what if we